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The present invention relates in general to an 
image processing technique and, more particularly, 
to a technique of aligning a plurality of picture 
images for image synthesis. 

Recently, it has become important to synthe- 
size a plurality of picture images, for example, two 
images picked up at different times. Some image 
processes apply to a medical imaging system such 
as a computed tomography (CT) scanner, which 
obtains an image of a selected plane section of a 
human body. One of these image processes ob- 
tains the difference between two images of a re- 
gion of interest (e.g., 
lected plane section in 
a change in such a re 



picture image synthesis, requires the precise align- 
ment of two images to produce a subtraction im- 
age. Poor image alignment cannot result in a good 
subtraction image. 

The simplest image alignment is based on the 
visual judgement of a human being. This alignment 
involves the comparison of two images, detection 
of misalignment (or mis-registration) between them, 
and transition of one of the images according to 
the misalignment. The image transition includes a 
simple coordinate conversion such as parallel tran- 

alignment technique is relatively effective when the 
misalignment, which is expressed in a vector quan- 
tity, is uniform over the entire image plane. A 
single picture image obtained for medical purposes 
(or a single satellite-transmitted picture image) is 
usually a complex of different sub-regions of an 
object so that the vector of the misalignment be- 
tween two such picture images is hardly uniform 
over the entire image plane. That is, regional vari- 
ations in misalignment vector often appear. In this 
case, the simple coordinate conversion technique 
cannot be effective in aligning two images over the 
entire image plane. 

There is a known method for solving the prob- 
lem. This method includes the steps of: 

1) Dividing both picture images of an object into 
several sub-image regions. 

2) Computing a correlation coefficient for each 
pair of corresponding sub-image regions of two 
picture images. 

3) Detecting the misalignment vector between 
that pair of sub-image regions which has the 
maximum correlation coefficient. 

4) Obtaining the misalignment over the entire 
image based on the detected vector. 
According to this method, however, when the 
misalignment between corresponding sub-image 

y small or a contrast dif- 



ation in the correlation coefficients is consider- 
ably small. This reduces the detection sensitivity 
of the misalignment vector and deteriorates 
alignment accuracy as a consequence. In this 
case, if the images have noise components, the 
significance of the variation in the correlation 
coefficients may be counteracted. Such an ad- 
verse effect of the noise greatly decreases the 
alignment accuracy and may result in alignment 

It is therefore an object of the present invention 
to provide a new and improved image processing 
technique for aligning, with higher accuracy, a pair 
of picture images such as those used for medical 
purposes or satellite-transmission. 

It is another object of the present invention to 
provide a new and improved self-alignment tech- 
nique for aligning a pair of such picture images 
with higher accuracy at higher speed. 

According to the image self-alignment system 
of the present invention, first, a desired image 
region is extracted from each of a pair of picture 
images to be aligned. Each image region is divided 
into n x m image segments arranged in a matrix 
form, where n and m are positive integers. There 
are two steps in detecting misalignment between 
each pair of image segments to be compared. The 
first one (coarse misalignment detection mode) 
uses a first evaluation parameter called a correla- 
tion coefficient for rough misalignment detection. 
The other (fine misalignment detection mode) uses 
a second evaluation parameter including data of 
the difference in pixel density for fine misalignment 
detection. Based on the final misalignment data 
between each image segment pair, which has been 
obtained with higher accuracy in these two modes, 
alignment between the associated two picture im- 
ages is automatically performed. 

The present invention is best understood by 
reference to the accompanying drawings, in which: 
Fig. 1 is a block diagram showing the overall 
configuration of an image alignment device ac- 
cording to one embodiment of this invention; 
Rg. 2 is a block diagram illustrating the interior 
of a partial image area designating unit provided 
in the image alignment device of Fig. 1 ; 
Fig. 3 is a block diagram depicting a detailed 
circuit ( onfiguration of a misalignment detecting 
unit provided in the image alignment device of 
Rg. 1; and 

Rg. 4 is a block diagram showing the interior of 
a misalignment correcting unit provided in the 
image alignment device of Fig. 1. 

1, an automatic image 
it system according to one embodiment of 
ow be described. Fig. 1 
he image alignment de- 



3 



EP 0 154 404 B1 



ciated with these units. 

In Fig. 1, a pair of picture images 10, 12, such 
as medical-purpose reconstructed tomographic im- 
ages of a person, is subjected to image alignment. 
First, these images 10, 12 are supplied to a partial 
image area designator 14 which specifies desired 
partial image areas 10a, 12a in the respective im- 
ages 10, 12. These partial image areas 10a, 12a 



relatively clear for detecting the overall misalign- 
ment of the image pair 10, 12. The "relatively clear 
characteristic" means a clearer difference in pixel 
density and higher contrast. For example, for medi- 
cal purposes, an image area of the contour of a 
bone would be considered better qualified as the 
one with such a characteristic than that of soft 
tissues of organs. 

According to this embodiment, the partial im- 
age areas 10a, 12a are specified by visual com- 
parison by an operator as follows. 

The partial image area designator 14 com- 
prises a display terminal 16 for image display (see 
Fig. 2). When the operator inputs an instruction to 
display the images 10, 12 via a keyboard 18, the 
images 10, 12 are read out from a image filing unit 
20 incorporating a recording medium such as a 
magnetic disk. The images 10, 12 are then trans- 
ferred to the display terminal 16 under the control 
of a display controller 22. As shown in Fig. 2, the 
Images 10, 12 to be aligned are simultaneously 
displayed on the screen of the display terminal 16. 
This facilitates visual comparison of those images 
by the operator. Then, the operator manipulates a 
specific key (not shown) on the keyboard 18 to 
move rectangular cursors K1 and K2 on the screen 
to desired locations (the partial image areas 10a, 
12a) on the images 10, 12. The movement of the 
cursors K1, K2 is controlled by a cursor controller 
24 connected to the keyboard 18. Each cursor has 



areas. Consequently, the areas on the picture im- 
ages 10, 12 occupied by the cursors K directly 
become the partial image areas 10a, 12a. This 
permits the operator to extract desired partial im- 
3S 10, 12 in real 



s 10a, 12a, 



The partial image ai 



plied to a processor unit 30 where each parti; 
image area is divided into a plurality of image unit 
or segments. For this function, the processor un 
30 is called a partial image area divider hereafte 
The image segments (n x m segments) of th 
partial image area 10a are expressed by "P„ m: 
and those of the partial image area 12a are e> 



each pair of corresponding image segments which 
are included in the respective partial image areas 
10a, 12a. 

Again in Fig. 1, a misalignment detecting unit 
40 is located at the subsequent stage of the partial 
image area divider 30. The misalignment detecting 
unit 40 comprises a segment pair designator 42 
and a misalignment calculating unit 44. The seg- 
ment pair designator 42 sequentially designates 
segment pairs, one from each partial image area 

(10a, 12a), e.g., Pn and Qn, P12 and Q12 Pij 

and Qij. The misalignment calculating unit 44 com- 
putes a misalignment vector between each des- 
ignated segment pair, for exmaple, P,j and Q B . This 
computation is based on an evaluation function with 
two different parameters, a correlation coefficient 
between the corresponding image segments P,j, Qij 
and a pixel density difference coefficient that is the 
sum of the absolute value of the difference in pixel 
densities between corresponding portions of the 
designated segment pair. (These parameters are 
hereafter called correlation data and pixel density 
difference data.) The misalignment vector for the 
first image segment pair, Pn and Qn, is cal- 
culated based on the evaluation function, and is 
supplied to a memory (to be described later). 
Then, the next image segment pair, e.g., P12 and 
Q12, is specified by the segment pair designator 
42. The misalignment vector for this segment pair 
is calculated by the calculating unit 44 and is 
stored in the memory. The same calculation pro- 
cess is executed for every one of the associated 
image segment pairs, thus providing a plurality of 
misalignment vector data for the partial image 
areas (10a, 12a). These plural pieces of misalign- 
ment vector data are supplied to an image position 
correcting unit 50. which extracts reliable ones 
from the vector data for overall alignment of the 
picture images 10. 12. This unit 50 can also par- 
tially correct the picture images with a divided 
picture segment of an image plane used as a unit, 
as desired. 

Fig. 3 is a detailed illustration of the interior of 
the misalignment detecting unit 40. The segment 
pair designator 42 is connected to an initial ad- 
dress memory 60 and address controllers 62, 64. 
The memory 60 stores center addresses of the 
partial image areas 10a, 12a. The address control- 
ler 62 (Pij address controller) stores data of the 
sizes (unit areas) of image segments P,j of the 
partial image area 10a and center addresses of the 
individual segments. The other address controller 
64 (Qij address controller) stores center address 
data and size data of image segments Q a of the 
partial image area 12a. 
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Based on the address data, the p g address control- 
ler 62 reads out the image picture from the first 
image memory 66 and extracts a specific image 
segment P5. The extracted image segment P a is 
stored in the image segment memory 68. The Q,j 
address controller 64 supplies the size data and 
central address data of an image segment Qij of 
the second image picture 12 to a neighboring re- 
gion designator 72 under the control of a coarse 
misalignment detecting circuit 70. The neighboring 
region designator 72 extracts an image segment 
with the specified size and the specified address 
being in the center from the image picture 12 
stored in a second image memory 74. The ex- 
tracted image segment Qrj is stored in a segment 
memory 76 (Q,, image segment memory) coupled 
to the second image memory 74. This pair of 
image segments P,j and Q t , respectively stored in 
the image segment memories 68, 76, is used for 
detection of the misalignment vector between the 
image pictures 10, 12 and position correction. 

Each of the image memories 68, 76 is coupled 
to calculators 80, 82. The calculator 80 sequentially 
calculates the correlation data, i.e., the first evalu- 
ation parameter, while the calculator 82 sequen- 
tially calculates the pixel density difference data 
(the second evaluation parameter). The image seg- 

memories 68, 76 is parallel-transmitted to those 
calculators 80, 82, which, in turn, respectively cal- 
culate the correlation data and pixel density dif- 
ference data for the image segment pair P,j and Q,j. 
The calculated correlation data and pixel density 
difference data are respectively stored in a correla- 
tion data memory 84 and a pixel density difference 
memory 86. This data process is repeated for 
neighboring regions around the image segment 
designated by the neighboring region designator 
72. That is, while shifting the image segment Q,, by 
a small amount on the image plane around the 
initial position, the correlation data between the 
image segment Qij and image segment Py - 
(stationary) is sequentially calculated. 

The correlation data calculator 80 and correla- 
tion data memory 84 are included in the coarse 
misalignment detecting circuit 70. The memory 84 
is coupled to a maximum detector 90, which de- 
tects the maximum correlation data among those 
stored in the memory 84 and generates an address 
signal 92 representing the center address of the 
image segment (Qij') having the maximum correla- 
tion data. The correlation coefficient between the 
Image segment P Ul and the image segment Qj' 
within the shifting range of the segment Q,j, is 

The address signal 92 is transmitted to the Q,j 
address controller 64 through a switch 94 that is in 
a first electrical mode as shown in Fig. 3. The Q,, 



address controller 64 supplies new center address 
data of the neighbouring image segment Qj' to the 
neighboring region designator 72 and a displace- 
ment calculator 96, which is coupled to the initial 

5 address memory 60 and the Qij address controller 
64. The displacement calculator 96 calculates the 
displacement between the center address of the 
initial image segment Q 0 , stored in the initial ad- 
dress memory 60, and the center address of the 

10 image segment Qj', represented by the signal 92 
obtained in the coarse misalignment detecting cir- 
cuit 70. The displacement or displacement vector 
data between the center addresses of the image 
segments Q g and Qij' is stored in a displacement 

15 memory 98. Based on the calculated displacement, 
the Qi, address controller 64 supplies the new 
center address data of the image segment Q g to 
the neighboring region designator 72. 

The coarse misalignment detecting circuit 70 

20 includes a switch controller 100; which detects the 
maximum (converged) correlation data among the 
all the correlation data obtained with the image 
segment Q,j being shifted around. At this time, the 
detecting circuit 70 controls the switch 94 to pre- 

25 vent the supply of the address signal 92 to the Q rj 
address controller 64. In the case where the cor- 
relation coefficients between the image segment 
pair P,j, Q,j are sequentially calculated for the seg- 
ment Qij being shifted, a gradual decrease in a 

30 change in the calculated correlation data can mean 
that the correlation data is approaching (or con- 
verging with) the maximum correlation data. When 
the variation in the correlation data over a period of 
time tails below a predetermined threshold level, 

35 i.e., when the maximum correlation data is con- 
verged, the switch controller 100 switches the 
switch 94 to end the coarse misalignment detection 
mode. The switch controller 100 serves to maintain 
the coarse misalignment detection (first misalign- 

40 ment detection) for the image segment pair P,, Qij 
as long as the change in the correlation data is 
above the threshold level. When the change in the 
correlation data for the image segment pair P,j, Q,j 
decreases to fall below the threshold level, the 

45 coarse misalignment detection is completed and 
the fine misalignment detection (second misalign- 
ment detection) starts. In this embodiment, the fine 
misalignment detection further uses the pixel den- 
sity difference data (the second evaluation param- 

50 eter). The following description discusses the ar- 
rangement of the switch controller 100 and explains 
the fine misalignment detection. 

A variation calculator 102 is connected to the 
maximum detector 90 to sequentially calculate 

55 temporal changes in the maximum coefficient data 
for the associated image segment pair. The ob- 
tained data of the temporal changes in the maxi- 
mum coefficient data is supplied to a data com- 
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parator 104 connected to a threshold value gener- 
ator 106 that prestores the threshold value 108. 
The comparator 104 compares the reference 
threshold value 108 with the variation in the maxi- 
that is an actual value. When 



signal, the switch driver 110 switches the switch 94 
from the first electrical mode to a second electrical 
mode, which prevents the address signal 92 of the 
coarse misalignment detecting circuit 70 from 
passing the switch 94. This cc 



E(xy): 



Suppose the evaluation parameter or evaluation 
function used in the fine misalignment detection is 
represented by "P(xy)" which is given by: 
P(xy) = S(xy) - kx'E(xy), 

where S(xy): data obtained by normalizing 
the correlation data calculated 
in the correlation data calcula- 
tor 80, 

data obtained by normalizing 
the pixel density difference 
data, 

x, y: coordination on the image 

kx: weighing parameter (positive). 

The greater S(xy) or the smaller E(xy) is, the great- 
er thus the defined function P(xy) becomes. P(xy) 
has a maximum of 1 .0 only when the image seg- 
ment pair matches ideally. 

Two normalizer circuits 112 and 114 are re- 
spectively coupled to the correlation data calculator 
80 and pixel density difference data calculator 82. 
The normalizers 112, 114 serve to compute the 
normalized data S(xy) and E(xy) based on the data 
from the calculators 80, 82. The normalizers 112 
and 114 supply the data S(xy) and E(xy) to a 
subtractor circuit 116, which performs a 
on s(xy) and E(xy) to obtain the evaluati 
P(xy) for fine misalignment detection. Thus at 
P(xy) is stored, together with the center ad ' 
Y) of the corresponding neighboring region, in a 
memory 118 located at the succeeding stage of the 



This process of obtaining P(xy) is repeated 
wherein one image segment Q,j' of the image seg- 
ment pair, the coarse alignment detection of which 
has been completed, is finely shifted on the same 
image plane under the control of the Qj address 
controller. Plural pieces of evaluation function data 
P(xy) obtained in each process are sequentially 
stored in the memory 118. A second maximum 
detector 120 coupled to the memory 118 detects 
the maximum one, P(xy)max, among the stored P- 
(xy) and generates a second address signal 122 



presenting the center address of an image seg- 
snt Qij" which has yielded P(xy)max. This ad- 
3ss signal 122 is fed back to the Q a address 
ntroller 64 via the switch 94 in the second elec- 
;al mode. Like in the coarse misalignment detec- 
n mode, the displacement calculator 96 calcu- 



address and the center address of the image seg- 
ment Qij". 

The maximum detector 120 is connected to a 



124 



the th 



ation function data that is an actual value. When 
the change in the actual evaluation function data 
increases to rise above the threshold value, it can 
a final matching between the 
d image segments Py, Q-," has substan- 
tially occurred. Thus, the comparator 124 supplies 
a misalignment detection end signal 132 to a 
switch driver 130 connected to the switch 94. In 
response to the signal 132. the switch 94 changes 
to the original electrical mode from the second 
electrical mode. This completed the fine misalign- 
ment detection and makes the image processing 
device ready for the coarse misalignment detection 
for the next image segment pair, for example. P mJ 
and Q| +1J . 

The switch driver 130, operable in the fine 
alignment detection mode, is coupled to three 
transmission switches S1, S2, S3. The first data 
transmission switch S1 is provided between the 
correlation data memory 84 and final (converged) 
correlation data memory 140 having correlation 
data relating to each optimum Qij'. This memory 
140 exclusively stores final correlation data 
(converged in the coarse misalignment detection 
mode) with address data specifying the image seg- 
ment pair which has yielded the final correlation 
data. The second data transmission switch S2 is 
the pixel density difference data 
a final (converged) pixel density 
e data memory 142 having pixel density 
3,j". The memory 
142 exclusively stores converged pixel density dif- 
ference data, which is also affixed with address 
data specifying the image segment pair that has 
yielded that pixel density difference data. The third 
data transmission switch S3 is provided between 
the displacement memory 98 and final misalign- 
ment data memory 144. This memory 144 exclu- 
sively stores misalignment vector data finally yield- 
ed through the coarse and fine misalignment de- 
tection modes. This misalignment vector data is 
also affixed with address data specifying the image 
segment pair which has yielded that particular vec- 
tor data. These switches S1, S2, S3 become con- 
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ductive in response to the misalignment detection 
end signal 132 from the switch driver 130. There- 
fore, upon completion of the fine misalignment 
detection, all the final data is stored in the memo- 
ries 140, 142, 144. 

The misalignment detection for the next image 
segment pair, e.g., P,.ij and Q,.ij, is also carried 
out in two steps; namely, the coarse and fine 
misalignment detection modes. These two steps 
are repeated for every corresponding image seg- 
ment pair of the partial image areas 10a, 12a. As a 
result, regional misalignment vector data for every 

image segment pair (Pn, Qn), .... (Pj, Qi,) (P,™ 

Q„„,) can be obtained. 

The image position correcting unit 50 deter- 
mines an optimum alignment for the picture images 
10, 12 based on the regional misalignment quan- 
tities between these picture images, i.e., misalign- 
ment quantities between all the image segment 
pairs. The following is an explanation of the posi- 
tion correction executed by the unit 50. 

Fig. 4 is a detailed illustration of the interior of 
the position correcting unit 50. The position cor- 
recting unit 50 includes a reliable misalignment 
data search circuit 148 which searches plural 
pieces of misalignment vector data, obtained by 
repeating the misalignment detection for all the 
image segment pairs, for the one with the highest 
reliability. As shown in Fig. 4, the search circuit 148 
includes two sorting circuits 150, 152 which are 
respectively coupled to the converged correlation 
data memory 140 and the converged pixel density 
difference data memory 142. The sorting circuit 
150 receives plural pieces of converged or final 
correlation data for plural pieces of image segment 
pairs Pj, Q,j and sorts the correlation data in order 
from the largest to the smallest, thus providing a 
hierarchical sequential organization. Consequently, 
the correlation data is stored in a hierarchic mem- 
ory 154, located at the succeeding stage of the 
sorting circuit 150, in order from the largest to the 
smallest. This forms a hierarchic structure for the 
plural pieces of correlation data obtained for the 
corresponding image segment pairs Pj, Qjj of the 
partial image areas 10a, 12a. The other sorting 
circuit 152 receives plural pieces of converged or 
final pixel density difference data for the same 
plural pieces of image segment pairs P tj , Q tj and 
sorts the pixel density difference data in order from 
the smallest to the largest. Therefore, the pixel 
density difference data is stored in another hierar- 
chic memory 156, located at the succeeding stage 
of the sorting circuit 152, in order from the smallest 
to the largest. This forms a hierarchic structure for 
the plural pieces of pixel density difference data 
obtained for the corresponding image segment 
pairs P,,-, Q,j of the partial image areas 10a, 12a. 

An AND circuit 158, coupled to the hierarchic 



data and pixel density difference data stored in 
these memories for the ones which are within a 
specific high rank and specify a common image 

5 segment pair. Assume that the specific rank is the 
highest 5 data in the hierarchic memories 154 and 
156. If there exists an image segment pair that has 
both the correlation data and pixel density dif- 
ference data ranked at the top in those memories 

10 154, 156, this pair is considered to have yielded a 
misalignment vector with the highest reliability. Nat- 
urally, that image segment pair would be selected 
without reservation. If there exists an image seg- 
ment pair which has the correlation data ranked as 

75 the second best in the hierarchic memory 1 54 and 
the pixel density difference data ranked as the third 
in the hierarchic memory 156, this pair is also 
selected because the misalignment vector of such 
a pair is considered to have a relatively high re- 

20 liability. An image segment pair with the best cor- 
relation data or pixel density difference data but the 
other data being ranked below the fifth would not 
be selected. For example, an image segment pair 
with the best correlation data but with the seventh 

25 rank pixel density difference data would not be 
selected. A memory 160 coupled to the AND cir- 
cuit 158 exclusively stores the positions in the 
hierarchic memories 154, 156, of the thus selected 
image segment pair. This sorting manner provides 

30 an image segment pair or pairs with a higher 
correlation coefficient and a smaller sum of the 
absolute values of pixel density differences. It can 
be said that the misalignment vector data obtained 
for the selected image segment pair (or pairs) has 

35 a high reliability. 

A position-adjusting address controller 162 is 
coupled to the final misalignment data memory 144 
shown in Fig. 3. This controller 162 finds out the 
position data of the image segment pair (or pairs) 

40 having the highly-reliable misalignament data 
stored in the memory 160 and accesses the mem- 
ory 144 to extract the final or converged misalign- 
ment data that is actually calculated for the con- 
cerned image segment pair (or pairs). Based on 

45 the reliable misalignment data, the address control- 
ler 162, which is also coupled to the image memo- 
ries 66 and 74, compensates for the misalignment 
between the image pictures 10, 12. When the ex- 
tracted partial image areas 10a, 12a are regions of 

so interest for a medical examination, a fine misalign- 
ment compensation can be performed on those 
image areas 10a and 12a. When the misalignment 
between the original image pictures is considered 
uniform over the entire picture plane, the overall 

55 alignment of the image pictures 10, 12 can be 
accurately carried out based on the mean vector of 
the selected reliable misalignment data. 

According to the imaging system incorporating 
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the image alignment apparatus embodying the 
present invention, first, desired partial image areas 
10a, 12a, which interest an operator or include 
clear properties, are extracted from a pair of tar- 
gets, i.e., image pictures 10, 12. Each of the ex- 
tracted areas 10a, 12a is divided into a plurality of 
image segments arranged in a matrix form. An 
arbitrary pair of segments (P|j, Q,j) corresponding to 
each other is selected from among these image 
segments and the misalignment vector between the 
pair is then calculated. After the misalignment vec- 
tor calculation is completed for one image segment 
pair, a final misalignment vector is stored in the 
memory 144. Then, the same misalignment vector 
calculation is taken with the next image segment 
pair until all the image segment pairs are involved. 

It should be noted that the misalignment detec- 
tion for each pair of corresponding image seg- 
ments is performed in two steps. The first step is 
the coarse misalignment detection mode in which 
the detection is carried out based on the correlation 
data for the image segment pair calculated as one 
of the pair is shifted around on the image plane. 
The second step is the fine misalignment detection 
mode in which the detection is carried out based 
on the evaluation function P(xy) and further de- 
pending on the pixel density difference data be- 
tween the Image segment pair. These two steps, 
involving not only the correlation data but also the 
pixel density difference data as two evaluation pa- 
rameters, will yield significantly reliable final mis- 
alignment vector information. The coarse detection 
mode is switched to the fine mode when the vari- 
ation in the correlation data updated in the coarse 
misalignment detecting circuit 70 falls below a spe- 
cific reference level or threshold level. Such a level 
is set by the threshold value generator 106 in- 
cluded in the coarse misalignment detecting circuit 
70. 

It can be said that the matching of the image 
segment pair has been approximately detected at 
the time the correlation data between the pair 
reaches the threshold level. Therefore, the misalig- 
nment vector can be obtained by calculating the 
displacement between the center coordinations of 
the image segments. However, the present inven- 
tion does not stop the detection at this stage but 
goes further into a finer misalignment detection. A 
new evaluation function based on the correlation 
data and pixel density difference data is introduced 
in the fine detection mode, thus ensuring finer 
detection of alignment between the associated im- 
age segment pair. The above-discussed point is 
one important technical feature of the present in- 

Another important feature resides in that one 
piece of misalignment vector data with a signifi- 
cantly high reliability is selected from the plural 



pieces of misalignment vector data obtained by 
repeating the aforementioned detection procedures 
for every image segment pair. The correlation data 
and pixel density difference data of each image 

5 segment pair are separately sorted to search for an 
image segment pair (or pairs) with good char- 
acteristics in both data. The segment pair or pairs 
are extracted as being significantly reliable. As a 
result, an accuracy in detecting regional misalign- 

10 ment vectors between the partial image areas 10a, 
12a, i.e., regions of interest, can further be im- 
proved. Therefore, even when the misalignment 
vector between medical-purpose picture images or 
satellite-transmitted images is uniform over the en- 

15 tire image plane, accurate alignment can be re- 
alized for at least a part of a region of interest. 
Further, even when noise is included in the partial 
image areas 10a, 12a, the process of selecting 
image segment pairs with high reliability can ex- 

20 elude those with lower reliability resulting from 
noise. This provides a highly accurate and reliable 
image alignment process which is substantially free 
of adverse noise effects. This further improves im- 
age synthesis. Thus, the image position alignment 

25 technique of the present invention can significantly 
contribute to image synthesis used in medical pur- 
poses, satellite-image processing, etc. 

Although the present invention has been shown 
and described with reference to a particular em- 

30 bodiment, various changes and modifications which 
are obvious to any persons skilled In the art to 
which the invention pertains are deemed to lie 
within the scope of the invention. 

For example, the evaluation function for speci- 

35 tying the quantity of a misalignment may be a 
combination of other indices than the correlation 
coefficient and pixel density difference which also 
represent the quantity image alignment. In the 
aforementioned embodiment, the information of 

40 pixel densities is used for the correlation coefficient 
and pixel density difference. However, when a dis- 
tinctive common characteristic is obviously seen 
between two image pictures, the alignment process 
can be performed paying attention only to that 

45 characteristic. In addition, the misalignment vector 
quantity obtained for each pair of partial image 
areas with high reliability may be used directly to 
correct the image position of that particular pair, 
thus aligning the associated image pictures seg- 

50 merit by segment. 



Claims 

1. An image processing apparatus for aligning a 
pair of first and second picture images, com- 
prising first processor means (14) for extract- 
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ing a desired partial image area from each of 
the first and second images to be subjected to 
position alignment, and second processor 
means (30) for dividing each of said partial 
image areas into n x m image segments (P,j, 5 
Qj) arranged in a matrix form, where n and m 
are positive integers, characterized in that said 
apparatus further comprises: 
third processor means (40): 
in a first mode, for selecting an initial image io 
segment (Py) of the first image; 
for detecting misalignment between said initial 
image segment (Pj) and a number of image 
segments (Qj) of said second image using a 
first evaluation parameter based upon correla- js 
tion coefficient data to select an optimum im- 
age segment (Q|,-') having a given maximum 
correlation coefficient; 

in a second mode, for detecting misalignment 
between said initial image segment (P ri ) and a 20 
number of image segments finely shifted about 
said optimum image segment (Qij') using a 
second evaluation parameter based upon pixel 
density difference data and correlation coeffi- 
cient data to select the optimum image seg- 25 
ment pair (P 5 , Qij") having a maximum evalu- 
ation function (Pxy); 

for selecting a further initial image segment 
(Pj.j, j) and repeating said detection in said 
first and second mode; 30 
for sloring misalignment data for each (P rj ); and 
fourth processor means (50) for selecting at 
least one optimum image segment pair (Pj, 
Q,j") having the optimum misalignment data 
and for aligning said first and second picture 35 
images based upon said optimum misalign- 
ment data. 

The apparatus according to claim 1 , character- 
ized in that said third processor means com- 40 
prises: 

first calculator means (80) for sequentially cal- 
culating said correlation coefficient data, when 
said first detection mode is carried out, by 
shifting at least one of said corresponding im- 45 
age segment pairs around an initial position on 
an image plane in a manner such that each 
shifting from said initial position makes the 
image segments before and after the shifting 
have some overlapping portions therebetween; 50 

first maximum detector means (90) for deter- 
mining the maximum correlation data among 
plural pieces of said correlation data calculated 
for each pair of said corresponding image seg- 55 

The apparatus according to claim 2, character- 



ized in that said third processor means further 
comprises: 

mode controller means (100), connected to 
said first maximum detector means (90), for 
electrically storing a threshold value of said 
correlation data and switching the detection 
mode from said first detection mode to said 
second detection mode when a variation in the 
correlation data between said image segment 
pairs, sequentially detected, decreases to fall 
below said threshold value. 

4. The apparatus according to claim 3, character- 
ized in that said third processor means further 
comprises: 

second calculator means (82) for summing the 
magnitudes of pixel density differences for 
each of said image segment pairs in said sec- 
ond detection mode to produce said pixel den- 
sity difference data; 

normalizer means (112, 114), connected to 
said first calculator means (80) and second 
calculator means (82), for normalizing said cor- 
relation data and pixel density difference data; 

third calculator means (116), connected to said 
normalizer means (112, 114), for calculating 
said second evaluation parameter based on 
said normalized correlation data and pixel den- 
sity difference data. 

5. The apparatus according to claim 4, character- 
ized in that said third processor means further 
comprises: 

second maximum detector means (120) for 
updating said second evaluation parameter to 
determine the maximum among a plurality of 
said second evaluation parameters calculated 
for the particular image segment pair; 
mode termination means (124, 130), coupled 
to said second maximum detector means 
(120), for electrically storing a threshold value 
of said second evaluation parameter and termi- 
nating said second detection mode performed 
for the particular image segment pair by said 
third processor means (40) when said updated 
second evaluation parameter exceeds said 
threshold value; and 

fourth calculator means (96) for detecting cen- 
ter addresses of each of said optimum image 
segment pairs and calculating a misalignment 
between said center addresses. 

6. The apparatus according to claim 5, character- 
ized by further comprising fifth processor 
means (148) for sorting final correlation data 
and pixel density difference data, which are 
respectively obtained in said first and second 



optimum image segment pairs of said partial 
image areas, for searching ali of said particular 
and optimum image segment pairs for at least 
one image segment pair having both correla- 
tion data and pixel density difference data 
highly ranked and for calculating a misalign- 
ment vector between said at least one image 
segment pair. 



storing misalignment data for each (Pi,); and 
selecting at least one optimum image segment 
pair (Pq, Qij") having the optimum misalign- 
ment data and for aligning said first and sec- 
ond picture images based upon said optimum 
misalignment data. 



y (154) fo 



said particular image segment pairs in order 
from the largest to the smallest; 
a second hierarchic memory (156) for storing 
said plural pieces of final pixel density dif- 
ference data for all of said optimum image 
segment pairs in order from the smallest to the 

circuit means (158, 160, 162), coupled to said 

tracting at least one image segment pair with 
the correlation data and pixel density differ- 
ence data both being in a specific high rank 
and for specifying misalignment vector data of 
said at least one image segment pair. 

An image processing method for aligning a 
pair of first and second picture images, com- 
prising the steps of extracting a desired partial 
image area from each of the first and second 
picture images to be aligned, and dividing 
each of said partial image areas into n x m 
image segments (P,,, Q ( {l arranged in a matrix 
form, where n x m are positive integers, 
jrther comprising the steps 



in a first mode, selecting an initial image seg- 
ment (Py) of the first image; 
detecting misalignment between said initial im- 
age segment (Ppj) and a number of image 
segments (Qy) of said second image using a 
first evaluation parameter based upon correla- 
tion coefficient data to select an optimum im- 
age segment (Qy') having a given maximum 
correlation coefficient; 

in a second mode, detecting misalignment be- 
tween said initial image segment (Py) and a 
number of image segments finely shifted about 
said optimum image segment (Q a ') using a 
second evaluation parameter based upon pixel 
density difference data and correlation coeffi- 
cient data to select the optimum image seg^ 
ment pair (Py, Qij") having a maximum evalu- 
ation function (Pxy); 



of said n x m image segment pairs included in 
said partial image areas of said first and sec- 
ond images. 

). The method according to claim 9, character- 
ized by further comprising the steps of: 
separately sorting plural pieces of final correla- 
tion data and pixel density difference data ob- 
tained for all of said image segment pairs; 
searching said all of said image segment pairs 
for at least one image segment pair having 
those correlation data and pixel density dif- 
ference data which are both in a predeter- 
mined high rank; and 

specifying the misalignment quantity between 
said at least one image segment pair as highly 



specified misalignment quantity for aligning 



11. The method according to claim 9, character- 
ized by further comprising the steps of: 
sorting said final correlation data in order from 
the largest to the smallest to form a hierarchic 
structure for said correlation data; 
sorting said final pixel density difference data 



sity difference data; 

searching all of said image segment pairs for 
at least one image segment pair having those 
correlation data and pixel density difference 
data which are both in a predetermined high 



Appareil de traitement d'images pour aligner 



tielle desiree de chacune des premiere et se- 
conde images a soumettre a un alignement de 
position, et des seconds moyens de traitement 
(30) pour diviser chacune des dites zones 
d'image partielles en n x m segments d'image 
(Pij, Qij), disposees sous forme matricielle, ou 
n et m sont des entiers positifs, caracterise en 
ce que le dit appareil comporte de plus : 
des troisiemes moyens de traitement (40) : 
dans un premier mode, pour selectionner un 
segment d'image initial (P u ) de la premiere 

pour detecter le defaut d'alignement entre le 
dit segment d'image initial (P|j) et un certain 
nombre de segments d'image (Q,,-) de la dite 
seconde image en utilisant un premier parame- 
tre devaluation base sur des donnees de coef- 
ficient de correlation pour selectionner un seg- 
ment d'image optimal (Qj') presentant un coef- 



dans un second mode, pour detecter le defaut 
d'alignement entre le dit segment d'image ini- 
tial (Pi,) et un certain nombre de segments 
d'image Iegerement decales par rapport au dit 
segment d'image optimal (Q,,') en utilisant un 
second parametre d'evaluation base sur des 
donnees de differences de densites de pixel et 
des donnees de coefficient de correlation pour 
selectionner la paire de segments d'image op- 
timaux (P,j, Qij") presentant une fonction d'eva- 
luation maximale (Pxy) ; 
pour selectionner un segment d'image initial 
supplemental (P m , ,■) et en repetant la dite 
detection dans les dits premier et second mo- 



ments d'image optimaux (Pij, Q|j") presentant 
les donnees de defaut d'alignement optimal et 
pour aligner les dites premiere et seconde 
images sur la base des dites donnees de 
defaut d'alignement optimal. 



Appareil selon la 
en ce que les dits troisiemes moyens de traite- 
ment comportent : 

des premiers moyens de calcul (80) pour cal- 



; un plan d'image d'une maniere 
telle que chaque decalage a partir de la dite 
position initiale place les segments d'image 
avant et apres le decalage en recouvrements 

des premiers moyens (90) de detection d'un 
maximum pour determiner les donnees de cor- 
relation maximale parmi plusieurs blocs des 
dites donnees de correlation calculees pour 
chaque paire des dits segments d'image cor- 



1 selon la revendication 2, caracterise 
ue les dits troisiemes moyens de traite- 
>mportent de plus : 

>yens (100) de commande du mode, 
ix dits premiers moyens (90) de 
detection du maximum pour memoriser electri- 
quement une valeur de seuil des dites don- 
nees de correlation et pour commuter le mode 
de detection du dit premier au dit second 
mode de detection quand une variation dans 
les dites pai- 



Appareil selon la revendication 3, rai In. 
en ce que les dits troisiemes moyens de traite- 
ment comporte de plus : 
des seconds moyens (82) de calcul pour ajou- 
ter les amplitudes des differences de densites 
de pixel pour chacune des dites paires de 
segments d'image dans le dit second mode de 
detection pour produire les dites donnees de 
differences de densites de pixel ; 
des moyens (112, 114) de normalisation, 



is (80) di 

calcul et aux dits seconds moyens (82) de 



quand le dit premier 
de detection est execute, en decalant au 
I'une des dites paires de segments 



correlation et les dites donnees de differences 
de densites ; et 

des troisiemes moyens (116) de calcul, 
connectes aux dits moyens (112, 114) de nor- 
malisation, pour calculer le dit second parame- 
tre d'evaluation base sur les dites donnees de 
correlation normalisees et les dites donnees de 
differences de densites de pixel. 



5. Appareil selon la revendication 4, caracterise 
en ce que les dits troisiemes moyens de traite- 
ment comportent de plus : 
des seconds moyens (120) de detection d'un 
maximum pour mettre a jour le dit second 
parametre d'evaluation pour determiner le 



parametres d'evaluation calcules pour la paire 



10 



19 



EP 0 154 404 B1 



particuliere de segments d'image ; 
des moyens (124, 130) de terminaison de 
mode, couples aux dits seconds moyens (120) 
de detection du maximum, pour memoriser 
electriquement une valeur de seuii du dit se- 5 
cond parametre d'evaluation et pour terminer 
le dit second mode de detection execute pour 
ia paire particuliere de segments d'image par 
les dits troisiemes moyens de traitement (40) 
quand le dit second parametre d'evaluation io 
mis a jour excede la dite valeur de seuil ; et 
des quatriemes moyens (96) de calcul pour 
detecter les adresses du centre de chacune 
des dites paires de segments d'image opti- 
maux et pour calculer un defaut d'alignement 75 
entre les dites adresses de centre. 

Appareil selon la revendication 5, caracterise 
en ce qu'il comports de plus des cinquiemes 
moyens (148) de traitement pour trier les don- 20 
nees de correlation finales et les donnees de 

nues respectivement aans les aits premier et 
second modes de detection pour les dites pai- 
res particuliere et optimale de segments d'ima- 25 
ge des dites zones d'image partielles, pour 
chercher chacune des dites paires particuliere 
et optimale de segments d'image pour au 
moins une paire de segments d'image ayant a 
la fois des donnees de correlation et des don- 30 
ndes de differences de densites de pixel clas- 
sees dans un rang d'ordre eleve et pour cal- 
culer un vecteur de defaut d'alignement entre 
au moins la dite paire de segments d'image. 

Appareil selon la revendication 6, caracterise' 
en ce que les dits cinquiemes moyens (148) 
de traitement comportent : 
une premiere memoire hierarchique (154) pour 
memoriser la dite pluralite de blocs de don- -to 
nees de correlation finale pour I'ensemble des 
dites paires particulieres de segments d'image 
en ordre de la plus grande a la plus petite ; 
une seconde memoire hierarchique (156) pour 
memoriser la dite pluralite de blocs de don- 45 
nees de difference de densites de pixel finale 
pour I'ensemble des dites paires optimales de 
segments d'image en ordre de la plus grande 
a la plus petite ; et 

des circuits (158, 160, 162) couples aux dites so 
premiere et seconde memoires hierarchiques 
pour extraire au moins une paire de segments 
d'image, dont les donnees de correlation et les 
donnees de differences de densites de pixel 
sont en meme temps dans un rang particulier 55 
elev6 et pour determiner des donnees vecto- 
rielles de defaut d'alignement d'au moins une 
paire de segments d'image. 



8. Precede de traitement d'images pour aligner 
une paire de premiere et seconde images, 
comprenant des etapes d'extraire une zone 
d'image partielle desiree de chacune des pre- 
miere et seconde images a aligner, et de divi- 

en n x m segments d'image (Pjj, Q PJ ), dispo- 
sers sous forme matricielle, 0C1 n et m sont 
des entiers positifs, caracterise en ce qu'il 
comports de plus les etapes de : 
dans un premier mode, de selectionner un 
segment d'image initial (P,) de la premiere 

de detecter le defaut d'alignement entre le dit 
segment d'image initial (P,) et un certain nom- 
bre de segments d'image (Q,j) de la dite se- 
conde image en utilisant un premier parametre 
d'evaluation base sur des donnees de coeffi- 
cient de correlation pour selectionner un seg- 
ment d'image optimal (Q u ') presentant un coef- 
ficient de correlation maximum donne ; 
dans un second mode, de detecter le defaut 
d'alignement entre le dit segment d'image ini- 
tial (P|j) et un certain nombre de segments 
d'image legerement decales par rapport au dit 
segment d'image optimal (Qj') en utilisant un 
second parametre d'evaluation base sur des 
donnees de differences de densites de pixel et 
des donnees de coefficient de correlation pour 
selectionner la paire de segments d'image op- 
timaux (Pi JP Qij") presentant une fonction d'eva- 
luation maximale (Pxy) ; 
de selectionner un segment d'image initial 
supplemental P it1 , j) et en repetant la dite 
detection dans les dits premier et second mo- 
de stocker des donnees de defaut d'aligne- 

de selectionner au moins une paire de seg- 
ments d'image optimaux (P,j, Qi,") presentant 
les donnees de defaut d'alignement optimal et 
pour aligner les dites premiere et seconde 
images sur la base des dites donnees de 
defaut d'alignement optimal. 

9. Precede selon la revendication 8, caracterise 
en ce qu'il comports de plus une etape de 
realisation repStee des dites premiere et se- 
conde detections de defaut d'alignement pour 
les paires de segments d'image autres que la 
dite paire selectionnee de segments d'image, 
fournissant ainsi des donnees vectorielles de 
defaut d'alignement final pour toutes les dites 
paires de segments d'image indues dans les 
dites zones d'image partielles des dites pre- 
miere et seconde images. 

10. Precede selon la revendication 9, caracterise 



en ce qu'il comporte de plus les etapes de : 
trier separement une pluralite de blocs de don- 
nees de correlation finale et des donnees de 



toutes les dites paires de segments d'image ; 
chercher chacune des dites paires de seg- 
ments d'image pour au moins une paire de 
segments d'image presentant ces donnees de 
correlation et ces donnees de differences de 
densites de pixel qui sont chacune a un rang 
eleve predetermine' ; et 

specifier la quantite de defaut d'alignement 
entre au moins une des dites paire de seg- 
ments d'image comme donnees de defaut 
d'alignement hautement fiables et pour utiliser 
la dite quantite de defaut d'alignement speci- 



I. Procede selon la revendication 9, caracterise 
en ce qu'il comporte de plus les Stapes de : 
trier les dites donnees finales de correlation 
dans I'ordre de la plus grande a la plus petite 
pour former une structure hierarchique des di- 
tes donnees de correlation ; 
trier les dites donnees finales de differences 
de densitSs de pixel dans I'ordre de la plus 
petite a la plus grande pour former une struc- 
ture hierarchique des dites donnees de diffe- 
rences de densites de pixel ; 
chercher chacune des dites paires de seg- 
ments d'image pour au moins une paire de 
segments d'image presentant ces donnees de 
correlation et ces donnees de differences de 
ont toutes a un rang 



specifier la quantite de defaut d'alignement 
entre au moins la dite paire de segments 
d'image comme donnees de defaut d'aligne- 
ment hautement fiables et pour utiliser la dite 
quantite de defaut d'alignement specifiee pour 
aligner les dites premiere et seconde images. 



Bildverarbeitungssystem zur Ausrichtung eines 
Paars von ersten und zweiten Bildern, mit ei- 
ner ersten Prozessoreinheit (14) zum Auszie- 
hen bzw. Aussieben eines gewiinschten Teil- 
bildbereichs aus jedem der ersten und zwei- 
ten, der Lagenausrichtung zu unterwerfenden 
Bilder und einer zweiten Prozessoreinheit (30) 
zum Teilen Oder Dividieren jedes der Teilbild- 
bereiche in n x m Bildsegment (Pij, 0%, die in 



eine dritte Prozessoreinheit (40), urn 
in einem ersten Modus ein Anfangsbildseg- 
ment (Pyj des ersten Bilds zu wahlen, 
urn eine Fehlausrichtung zwischen dem An- 
fangsbildsegment (P|j) und einer Anzahl von 
Bildsegmenten (Qij) des zweiten Bilds unter 
Benutzung eines ersten Bewertungsparameters 
auf der Grundlage von Korrelationskoeffizient- 
daten fur das Wahlen eines optimalen Bildseg- 
mentes (Q,-) el 
relationskoeffizienten zl 
urn in einem zweiten Modus eine Fehlausrich- 
tung zwischen dem Anfangsbildsegment (P r j) 
und einer Anzahl von urn das optimale Bild- 
segment (Q,j) feinverschobenen Bildsegmenten 
unter Benutzung eines zweiten Bewertungspa- 
rameters auf der Qrundlage von Pixeldichte- 
Differenzdaten und Korrelationskoeffizientdaten 
fur das Wahlen des optimalen Segmentpaars 
(P,j, Qij') mit einer maximalen Bewertungsfunk- 
tion (Pxy) zu detektieren, urn ein weiteres An- 
fangsbildsegment (Pm, j)zu wahlen und die 
Detektion in erstem und zweitem Modus zu 
wiederholen, sowie urn Fehlausrichtungsdaten 
fiir jedes Segment (P rj ) zu speichern, und 
eine vierte Prozessoreinheit (50) zum Wahlen 
mindestens eines optimalen Bildsegmentpaars 
(Pij, Qij") mit den optimalen Fehlausrichtungs- 
daten und zum Ausrichten der ersten und 
zweiten Bilder auf der Qrundlage der optimalen 
Fehlausrichtungsdaten. 



eine erste Recheneinheit (8 

bei Durchfuhrung des ersten Detektionsmodus 
durch Verschieben mindestens eines der be- 
treffenden Bildsegmentpaare urn eine Anfangs- 
position auf einer Bildebene herum in der Wei- 
se, daB bei jeder Verschiebung aus der An- 

s Oberlappungsab- 
schnitte zwischen sich aufweisen, und eine er- 
ste Maximumdetektoreinheit (90) zum Bestim- 
men der maximalen Korrelationsdaten unter ei- 
ner Vielzahl von Einheiten der fur jedes Paar 



3. System nach Anspruch 2, dadurch gekenn- 
zeichnet, da/3 die dritte Prozessoreinheit ferner 

eine mit der ersten Maximumdetektoreinheit 
(90) verbundene Modussteuereinheit (100) zum 
elektrischen Speichern eines Schwellenwerts 
der Korrelationsdaten und zum Umschalten 



12 



23 
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n ersten Detektions- 
modus auf den zweiten Detektionsmodus, 
wenn eine Anderung in den Korrelationsdaten 
zwischen den Bildsegmentpaaren, sequentiell 
detektiert, abnimmt und unter den Schwellen- 
wert fallt. 

System nach Anspruch 3, dadurch gekenn- 
zeichnet, daB die dritte Prozessoreinheit ferner 
umfafit: 

eine zweite Recheneinheit (82) zum Summie- 
ren der GroBen der P ' 



auch Pixeldichtedifferenzdaten ( 





(112, 114) zum Normieren der Korrelationsda- 
ten und der Pixeldichtedifferenzdaten sowie 
eine mit der Normiereinheit (112, 114) verbun- 
dene dritte Recheneinheit (116) zum Berech- 
nen des zweiten Bewertungsparameters auf 
der Grundlage der normierten Korrelationsda- 
ten und Pixeldichtedifferenzdaten. 



eine zweite Maximumdetektoreinheit (120) zum 
Aktualisieren des zweiten Bewertungsparame- 
ters zwecks Bestimmung des Maximums unter 
elner Vielzahl der fur das spezielle Bildseg- 

eine mit der zweiten Maximumdetektoreinheit 
(120) gekoppelte Modusbeendigungseinheit 
(124, 130) zum elektrischen Speichern eines 
Schwellenwerts des zweiten Bewertungspara- 
meters und zum Beendigen des fur das spe- 
zielle Bildsegmentpaar durch die dritte Prozes- 
soreinheit (40) durchgefuhrten zweiten Detek- 



einen ersten hierarchischen Speicher (154) 
zum Speichern der zahlreichen Einheiten end- 
giiltiger Korrelationsdaten fUr alle speziellen 
Bildsegmentpaare in der Reihenfolge von den 
groSten zu den kleinsten Daten, 
einen zweiten hierarchischen Speicher (156) 
zum Speichern der zahlreichen Einheiten der 
endgUltigen Pixeldichtedifferenzdaten fur alle 
optimalen Bildsegmentpaare in der Reihenfol- 
ge von den kleinsten zu den groBten Daten 

eine mit erstem und zweitem hierarchischen 
Speicher gekoppelte Schaltungseinrichtung 
(158, 160, 162) zum Ausziehen bzw. Aussie- 
ben mindestens eines Bildsegmentpaars, bei 
dem die Korrelationsdaten und die Pixeldichte- 
differenzdaten jeweils mit einem spezifischen 
hohen Rang vorliegen, und zum Bezeichnen 
von Fehlausrichtungsvektordaten des minde- 
stens einen Bildsegmentpaars. 



8. Bildverarbeitungsverfahren z 

nes Paars von ersten und zweiten biiaern, 
umfassend die Schritte des Ausziehens bzw. 
Aussiebens eines gewUnschten Teilbildbe- 
reichs aus jedem der auszurichtenden ersten 
und zweiten Bilder und des Teilens Oder Divi- 
dierens jedes der Teilbildbereiche in n x m 
Bildsegmente (Pij.Qij), die in einer Matrixform 
angeordnet sind, wobei n und m positive gan- 
ze Zahlen darstellen, dadurch gekennzeichnet, 



40 daB 



nt (P 5 ) des er 



x Anzahl vi 



Bild- 



Fehlausrichtung 



System nach Anspruch 5, gekennzeichnet 
durch eine fUnfte Prozessoreinheit (148) zum 
Sortieren endgultiger Korrelationsdaten und Pi- 
xeldichtedifferenzdaten, die jeweils in erstem 
bzw. zweitem Detektionsmodus fur die speziel- 
len und optimalen Segmentpaare der Teilbild- 
bereiche gewonnen Oder ermittelt werden, zum 
Absuchen alter speziellen und optimalen Bild- 
segmentpaare nach mindestens einem Bild- 
it sowohl Korrelationsdaten als 



bildsegment (P,j) und 
segmenten (Qij) 
zung eines ersten Bewertungsparameters auf 
der Grundlage von Korrelationskoeffizientdaten 
fur das Wahlen eines optimalen Bildsegmentes 
(Q,/) eines gegebenen maximalen Korrelations- 
koeffizienten detektiert wird, 
in einem zweiten Modus eine Fehlausrichtung 
zwischen dem Anfangsbildsegment (P,) und 
einer Anzahl von urn das optimale Bildsegment 
(Qij') feinverschobenen Bildsegmenten untei 
Benutzung eines zweiten Bewertungsparame- 
ters auf der Qrundlage von Pixeldichte-Diffe- 
renzdaten und Korrelationskoeffizientdaten fUr 
das Wahlen des optimalen Segmentpaars (P u , 
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Qj,-') mit einer maximalen Bewertungsfunktion 
(Pxy) detektiert wird, ein weiteres Anfangsbild- 
segment (P 1+ i, j) gewahlt und die Detektion in 
erstem und zweiten Modus wiederholt werden, 
Fehlausrichtungsdaten fur jedes Segment (P,j) 5 
gespeichert werden und 
mindestens ein optimales Bildsegmentpaar (P|j, 
Qj") mit den optimalen Fehlausrichtungsdaten 
gewahlt und die ersten und zweiten Biider auf 
der Grundlage der optimalen Fehlausrichtungs- w 
daten ausgerichtet werden. 



segmentpaar als hochst zuverlassige Fehlaus- 
richtungsdaten bezeichnet und die bezeichnete 
FehlausrichtungsgroBe fur die Ausrichtung der 
ersten und zweiten Biider benutzt werden. 



Verfahren nach Anspruch 8, dadurch gekenn- 
zeichnet, dafl die ersten und zweiten Fehlaus- 
richtungsdetektionen fur das vom gewahlten 
Bildsegmentpaar verschiedene Bildsegment- 



10. Verfahren nach Anspruch 9, dadurch gekenn- 
zeichnet, daB zahlreiche Einheiten von fur aile 
der Bildsegmentpaare erhaltenen endgultigen 
Korrelationsdaten und Pixeldichtedifferen2da- 
fsn getrennt sortiert werden, alle der Bildseg- 
mentpaare auf mindestens ein Bildsegment- 
paar mit diesen Korrelationsdaten und Pixel- 
dichtedifferenzdaten, die beide in einem vorbe- 



werden und 



destens einen Bildsegmentpaar als hBchst zu- 
verlassige Fehlausncntungsaaten oezeichnet 
' ' te FehlausrichtungsgroBe fUr 



I. Verfahren nach Anspruch 9, dadurch gekenn- 
zeichnet, da/J die endgultigen Korrelationsda- 
ten in der Reihenfolge von den grbSten zu den 
kleinsten Daten sortiert werden, urn eine hier- 
archische Struktur fur die Korrelationsdaten zu 
bilden, 

die endgultigen Pixeldichtedifferenzdaten in 
der Reihenfolge von den kleinsten zu den 
grfifiten Daten sortiert werden, urn eine hierar- 
chische Struktur fur die Pixeldichtedifferenzda- 



ein Bildsegmentpaar mit diesen Korrelationsda- 
ten und Pixeldichtedifferenzdaten, die beide in 
einem vorbestimmten hohen Rang vorliegen, 
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FIG. 4 
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